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planned by the OEP. ( SLD) 



* ★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★-A-* 

* Reproductions supplied by EDRS are the best that can be made * 

* from the original document . * 

******************************************************************************** 



ERIC 



TM029307 











"V. 



* nf Ji s PJ p ARTMENT OF EDUCATION [ 

~ omce of Educational Research and Improvement I. 

: EDUCATIONAL RESOURCES INFORMATION 

. S' CENTER (ERIC) 

p This document has been reproduced as 
received from the person or organization 
originating it. 

□ Minor changes have been made to 
.! improve reproduction quality. 

Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 



; permission to reproduce and 

l j DISSEMINATE THIS MATERIAL 
HAS BEEN GRANTED BY 

1 fern/ is 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



Publication Number 98.01 





>i 






98.01 



Agenda 1998-99 



Preface 

The Office of Program Evaluation (OPE) is charged with evaluating federally, 
locally, and state funded programs in the Austin Independent School District (AISD). The 
function of OPE has shifted over the past years. While continuing to carry out mandated 
reporting for federal and state grants, OPE now reports to the Deputy Superintendent of 
Instructional Services and School Operations. OPE staff work closely with program staff and 
increasingly carry out formative evaluation designed for program improvement as well as 
continuing to carry out summative evaluation designed to help district decisionmakers make 
programmatic decisions. 

A program evaluation agenda for AISD is prepared before the beginning of each 
school year to obtain agreement among decisionmakers that the proper and most critical 
information needs are being addressed. The agenda is developed through an interactive 
process involving the staff of OPE, the staff of special programs, the Superintendent of 
schools, his cabinet, and other AISD personnel. 

The Agenda describes methodology of the planned evaluations for each school year. 
These plans provide the blueprints for the evaluation staff to follow throughout the school 
year. In addition to the evaluations and activities outlined in this document, OPE will 
conduct or assist with projects, as needed, during the year. These special projects and ad hoc 
requests are not represented by an overview. 

The format of an evaluation overview is described below. Overviews for each 
planned evaluation and planned activity are included in this document and are presented in 
the following format: 

1. A heading, which gives the name of the program, the person(s) to contact about the 
program, and the person(s) to contact about the evaluation or activity; 

2. A brief program description, which provides general information about the 
program, the features of the program, its goals and objectives, and other information 
pertinent to understanding its importance to the district; 

3. Evaluation objectives, which enumerate the reason(s) the evaluation or activity is 
being performed; and 

4. Scope and method, which delineate the breadth of the evaluation or activity and 
the methods by which relevant data will be collected and analyzed. 

Readers of the agenda are encouraged to direct their comments and questions about 
the planned evaluations and activities included in this document to the OPE contact persons 
named in the overviews. 
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1998-99 Program Evaluation Summary 



i Evaluation/ 

Supporting Activities 


Mandated 

Evaluation 

• 


Reporting 

Period 


| Funding 
Source 

i - 


j Program 

| { Contact 


i OPE 

j Staff 


OPE 

j Supervisor 


Evaluations 


Title I Programs 


Title I-A (Regular) 


✓ 


Annual 


Federal 


Rollie Ford 


JC/EP/ 

WW 


GZ 


Schoolwide Programs 


✓ 


Annual 


Federal 


Rollie Ford 


WW/JC 


GZ | 


Pre-K/K 


✓ 


Annual 


Federal 


Jacquie Porter 


JC 


GZ 


Parental Involvement 
School Support 


✓ 


Annual 


Federal 


Rollie Ford 


WW 


GZ 


Neglected Facilities 


✓ 


Annual 


Federal 


Mary Thomas 


EP 


GZ 


Nonpublic Schools 


✓ 


Annual 


Federal 


Mary Thomas 


EP 


GZ 


Other Title I Programs 


✓ 


Annual 


Federal 


Rollie Ford 


GZ 


GZ 


Title I-C (Migrant) 


✓ 


Annual 


Federal 


Della Moore 


WW 


GZ 


Title I-D 

(Delinquent-Formula) 


✓ 


Annual 


Federal 


Mary Thomas 


EP 


GZ 


Extended Year Pro p ™ me 


Optional Extended Year 


✓ 


Annual 


State 


Mary Thomas 


WW 


GZ 


Intersession 


✓ 


Annual 


Federal 


Linda Bayley/ 
Susan Kemp 


WW 


GZ 


Summer School 


✓ 


Annual 


Federal 


Terry Ross 


JC/WW 


GZ 


! Professional Development 




Literacy & Mathematics at 
Read 




Special 

Project 


Local 


Darlene 

Westbrook 


JC/WW 


GZ 


Title VI 

(Innovative Programs) 


✓ 


Annual 


Federal 


Terry Ross 


EP 


GZ 


Administrative & Teacher 
Appraisal Systems 




Special 

Project 


Local 


Paul Shooter 


HK 


HW 
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Evaluation/ 

SUPPORTING ACTIVITIES 


Mandated 

Evaluation 


Reporting 

Period 


Funding 

Source 


Program 

Contact 


OPE 

Staff 


| OPE 

Supervisor 


Bilingual Programs 


I Bilingual Education 


✓ 


Annual 


State 


Della 

Moore 


RG 


RS 1 


I Title VII-A (Dual Language 
1 Project) 


✓ 


Annual 


Federal 


Della 

Moore 


RG 


RS 


Title VII-C 
(Immigrant) 


✓ 


Annual 


Federal 


Della 

Moore 


RG 


RS | 


Rethinking Education for 
.. M ; u.o.dtv....S.t.] identsIYII- A.) 


✓ 


Annual 


Federal 


Della 

Moore 


RG 


RS 


| Safe And Drug Free Schools & Communities and Alternative Programs 


l 




Title IV Programs: SDFSC 
Formula 


✓ 


Annual 


Federal 


Mary 

Walker 


MD/SS 


MD | 


ROPES 


✓ 


Annual 


Federal 


Bill 

Perry 


SS 


MD 1 


Positive Families 


✓ 


Annual 


Federal 


Joann 

Farrell 


MD/SS 


MD } 


SUPER I 


✓ 


Annual 


Federal 


Joe 

Oliveri 


MD/SS 


MD j 


SAP 


✓ 


Annual 


Federal 


Stan 

Brein 


SS 


MD 


PAL 


✓ 


Annual 


Federal 


Richard 

Sutch 


SS 


MD 


Middle School Pregnant and 
I Parenting Program 


✓ 


Annual 


Federal 


Gloria 

Williams 


HK 


HW ‘ 


Novanet 




Annual 


Local 


Mollie Guion 


KW 


RS 


Families Communicate 




Annual 


Local 


Joann Farrell 


MD 


RS 


j School to Career "PffS 


. 




- .fe: . 


sa mi ; 


School to Work 




Special 

Project 


Local 


TBA 


EP 


GZ 


Carl Perkins 


✓ 


Annual 


Federal 


TBA 


EP 


GZ : 


College Bound (AP, Merit 
Scholar) 




Special 

Project 


Local 


Amanda 

Batson 


EP 


ozj 
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Evaluation/ 
Supporting Activities 


Mandated 

Evaluation 


Reporting 

Period 


j Funding ; 

1 Source 

i : 


Program 

Contact 


OPE 

Staff 


OPE 

Supervisor 


Other Programs 


Academics 2000 


✓ 


Annual 


Federal 


Susan 

Kemp 


HK 


HW 


Administrative Support 
Services 




Special 

Project 


Local 


AC Gonzalez 


HW 


HW 


ExceL 




Annual 


Local 


Amanda 

Batson 


HK 


HW 


Inclusion in Middle & High 
School 




Special 

Project 


State 


Julie Lyons 


Consultant 


HW 


Local Accountability System 




Special 

Project 


Local 


Kay Psencik 


HW 


HW 


National Science Foundation 
Grant (NSF) 


✓ 


Annual 


Federal 


Laurie Mathis 


MB 


HW 


TAAS Analysis 




Annual 


Local 


Darlene 

Westbrook 


HW 


HW j 




Other Activities 


■ i 


Ad hoc requests/Special 
Projects 




Ongoing 


Local 


N/A 


HK/EA 


HW/RS 


Coordinated Survey 




Annual 


Local 


N/A 


KW 


RS 


GENESYS 




Annual 


Local 


Veda Raju 


Veda 

Raju 


RS 




Internal Data Requests 




Ongoing 


Local 


N/A 


KW 


RS/HW 


Procedures Manual 




Special 

Project 


Local 


N/A 


Evaluation 

Analysts 


HW 


SAS 




Ongoing 


Local 


N/A 


HK 


HW 


Web Page 




Updated 

Annually 


Local 


N/A 


RS 


RS 
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Staff Key: 



MB 


Michelle Batchelder 


HK 


Holly Koehler 


WW 


Wanda Washington 


JC 


Janice Curry 


EP 


Ertha Patrick 


HW 


Holly Williams 


MD 


Martha Doolittle 


RS 


Ralph Smith 


KW 


Karen Wendt 


RG 


Rosa Gonzalez 


SS 


Sue Sharkey 


GZ 


Gloria Zyskowski 
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Title I Program 



Program Director: 
Evaluation Supervisor: 
Evaluation Staff: 



Rollie Ford 
Gloria Zyskowski 
Janice Curry, Ertha Patrick, 
Wanda Washington 



Program Description 

Title I is a compensatory education program supported by funds from the 
Department of Education through the Elementary and Secondary Education Act of 1965 as 
amended by the Improving America’s Schools Act of 1994 (P.L. 103-382). The purpose of 
Title I is to enable schools to provide opportunities for children served to acquire the 
knowledge and skills described in the state content standards and to meet the state 
performance standards developed for all children. 

The level of Title I funding for a district is based on the percentage of low-income 
students living in the district attendance area. Title I funding for a school is determined by 
the percentage of low-income students living in the school’s attendance area. For district 
purposes, a child is defined as low- income if he or she receives free or reduced lunch. Schools 
are ranked in the spring of each year on the basis of the percentage of low-income children 
residing in the school’s attendance area. Districts must serve schools with 75% or more low- 
income students residing in their attendance area. Remaining schools that are below 75% low 
income are served in rank order as funding allows. 

The 1998-99 AISD Title I allocation, including roll-forward funds, is $10,806,376. 
AISD uses this sum to fund 50 schoolwide programs: 43 elementary schools and 7 secondary 
schools. This number includes all of the schools with 60% or more low-income students, and 
it represents a change from previous years in which the threshold for Title I funding was 70% 
or more low-income students. 

The programs that are funded by Title I and evaluated by the evaluation staff during 
the 1998-99 school year are: 

1 . Schoolwide Programs (SWPs) 

2. The Prekindergarten (pre-K) Program 

3. The Parental Involvement Component 

4. Nonpublic School Programs 

5. Neglected Facility Programs 

6. Delinquent Institution Programs 

7. Extended Year Programs (year-round schools, summer school, etc.) 

Schoolwide Programs 

As a result of the reauthorization of Title I in 1994, a school can be a Title I 
schoolwide program if 50% of the children in the school’s attendance zone or 50% of the 
children enrolled in the school are low-income students. Because AISD is serving students at 
schools that are at or above the 60% low income level in 1998-99, each of the 50 AISD 
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Title I schools is a schoolwide program. All students at a schoolwide program are considered 
eligible for Title I assistance. Schoolwide programs have a great deal of flexibility in using 
federal education funds, subject to rules established by the Department of Education. The 
spirit of the law is cooperation among funding sources and inclusion of all students. 

The Prekindergarten Program 

In the 1998-99 school year, prekindergarten (pre-K) programs at 54 AISD schools 
will continue to serve students who have been identified as economically disadvantaged, 
limited-English-proficient, or homeless. Half-day prekindergarten is mandated and funded by 
the state for all four-year-olds who meet one of these criteria. Additional instructional time 
for students is offered through the full-day pre-K program funded by Title I at elementary 
schools with the highest concentrations of low-income students. During 1998-99, 34 of the 
43 Title I elementary schools will provide full-day prekindergarten. 

The Parental Involvement Component 

Title I, Title I Migrant, and the district fund the Parent and Parent-Community 
Involvement components. The common goal of these programs is to build partnerships that 
benefit not only students and parents, but schools and communities as well. Combined funds 
are used to employ a parent program specialist and parental involvement representatives. 
These staff members provide workshops for parents and community members on various 
topics, make home visits, translate information when necessary, provide adult literacy 
training, and generally do all that they can to support parental and community involvement 
in the schools. 

Nonpublic Schools 

Eight nonpublic schools in the AISD attendance area received Title I funds in 1 997- 
98. These schools offer additional instructional services to low-income students in 
prekindergarten through grade 8. In 1998-99, the number of nonpublic schools receiving 
Title I funds is likely to increase. 

Neglected or Delinquent Facility Program 

Two neglected facilities and five delinquent institutions will receive funds from Title I 
in 1998-99: Settlement Home, Youth Options, Gardner-Betts (Travis County Juvenile 
Detention Facility), The Oaks Psychiatric Health System, Phoenix Academy of Austin, 
Turman House, and Travis County Leadership Academy. Title I funds will be used primarily 
to provide compensatory reading and mathematics services to youth at these facilities. 

Extended Year Programs 

In 1998-99, the year-round school calendar will be used in 11 Title I elementary 
schools. In this program, the school year revolves around an approximate 60/20 schedule 
(60 days in school and 20 days out) in contrast to the traditional nine-month calendar. The 
breaks between the 60-day sessions are called intercessions. Students falling behind in 
achievement are provided supplementary instruction during these intersessions. Federal funds 
are used for salaries, materials, and costs associated with support staff needed during the 
intersessions. 
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Summer Opportunity to Accelerate Reading (S.O.A.R.) is the district’s Title I 
summer school. It provides early intervention to accelerate the literacy learning of 
students entering grades 1-3 in the fall of 1998, and is designed to help achieve the district 
and state goal of insuring that all children will read on grade level by the end of grade 3. 

The optional extended year program is a supplemental grant program that 
provides additional instructional time for students who are at risk of academic failure. 
Although not funded by Title I, the program is included in this section because the grant is a 
funding source for extended year programs, such as summer school and intersessions. 

Title I Evaluation Objectives 

1. To comply with federal law requiring an annual evaluation of programs funded with 
Title I monies; and 

2. To provide information for decision makers on program effectiveness to facilitate 
decisions about program modification. 

Scope and Method 

The mandatory evaluation of the 1998-99 Title I program will be based on the 
Statewide Accountability System. Further evaluation will be performed at the district level. 
Qualitative as well as quantitative data will be gathered for evaluation of the Title I program. 

Schoolwide Programs 

Title I schoolwide programs will be evaluated by the same criteria as other schools in 
AISD and the state. These criteria include the following: percentage of students passing the 
Texas Assessment of Academic Skills (TAAS), annual dropout rate, and attendance rate. The 
data will be disaggregated by ethnicity, gender, English proficiency status, migrant status, and 
economic status. Only the students who are enrolled in the district as of October 30 and who 
remain in the district to take the TAAS in May will be included for accountability purposes. 
For 1998-99, the requirements for each criterion are as follows: 

1. At least 45% of all students at a campus must pass each section of the TAAS test, 
including reading, mathematics, and writing (grades four and eight only). Also, at 
least 45% of students in each disaggregated group must pass the TAAS. 

2. The annual dropout rate must be 6% or less for a middle school campus, and for each 
disaggregated group at the middle school campus. 

3. The attendance rate for a campus must be 94% or higher. 

For the 1998-99 school year, the Title I evaluation staff will undertake a study of 
professional development as it impacts instruction provided in Title I classrooms. For 
additional information on this aspect of the Title I evaluation, see the section in this 
document on the Professional Development Academy evaluation plan. 

Pre-K 

Language development is a major part of early childhood education in AISD. Because 
over 20% of the Title I budget is used to fund full-day pre-K, it is important to evaluate the 
pre-K program annually. The half-day programs give a comparison group for the evaluation. 
The language development of half- and full-day pre-K students will be assessed with the 
English and Spanish versions of the Peabody Picture Vocabulary Test. 
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A random sample of pre-K students will be pre- and posttested with the English 
language Peabody Picture Vocabulary Test-Ill (PPVT-III) in the fall and the spring. Spanish- 
speaking pre-K students will be pre- and posttested with the Spanish version of the PPYT, 
the Test de Vocabulario en Imagines Peabody (TVIP), as well as the PPVT-III. Overall 
gains from fall to spring on both tests will be examined for statistical significance. Other 
evaluation activities will include review of student demographics, as well as comparisons of 
achievement data for students in full-day and half-day pre-K programs, and for students in 
year-round and regular-calendar pre-K programs. 

The Title I evaluation staff will also work with the early childhood coordinators to 
provide training to pre-K teachers in the PPVT and TVIP assessments. The evaluation staff 
will provide information on how the tests are administered, how to interpret test results, and 
how to analyze scores in terms of response categories. 

Parent-Community Involvement 

Parent and community involvement in school activities such as attendance at 
meetings, workshops, and so on will be tracked. A survey of school staff and review of 
support documents will be used to investigate the effectiveness of these components. 

Private Schools/Neglected or Delinquent Institutions 

The number and demographic information of students served at the private schools 
and the institutions for neglected or delinquent students will be collected and reported to the 
Texas Education Agency (TEA). School staff will be surveyed at the end of the school year 
to provide qualitative data on implementation and effectiveness of services. For private 
schools, achievement data will be gathered and analyzed to investigate program effectiveness. 
In addition, Title I evaluation staff will provide training in the administration and 
interpretation of the PPVT for staff at the Title I-funded private schools to assist them with 
the evaluation of their pre-K students. 

Extended Year Programs 

Extended year programs will be evaluated using TAAS data and progress indicators 
based on the Statewide Accountability System criteria. Results for students who are served by 
extended year programs will be compared with results for similar students who are not served 
by extended year programs, and with results for students districtwide. 

The Title I evaluation staff will collaborate with teachers and program coordinators 
to gather data concerning the effectiveness of the Title I summer program, S.O.A.R. Student 
data, including pre- and posttest scores, will be examined. Staff data, including survey and 
interview responses, will also be included in the evaluation. 
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Title I Migrant Program 



Program Director: 
Evaluation Supervisor: 
Evaluation Staff: 



Della Moore 
Gloria Zyskowski 
Wanda Washington 



Program Description 

The Title I migrant programs are authorized under Part C of Title I of the 
Elementary and Secondary Education Act of 1965 as amended by the Improving America ’s 
Schools Act of 1994 (P.L. 103-382). State educational agencies (SEAs) receive funds for the 
costs of operating supplementary reading and mathematics education programs, including 
summer school, to benefit migrant children ages 3 through 21 (or until attainment of a high 
school diploma, whichever comes first). 

Under the law, major stipulations are (1) each recipient of funds “shall give priority 
to migratory children who are failing, or most at risk of failing, to meet the state’s 
challenging content standards and student performance standards, and whose education has 
been interrupted during the regular school year,” and (2) staff shall consult parents in 
planning, operating, and evaluating the program. 

The objective of the Title I Migrant Education Program is to support state and local 
programs that meet the special educational needs of the children of migratory workers. 
Under the program, the U.S. Education Department provides formula grants to states, which 
typically distribute most of the funds to local operating agencies (often a school district). 

The activities of the Migrant Program are centered around (1) recruitment of 
students, (2) parental involvement, and (3) a supplementary instructional program for 
secondary students. The NGS (New Generation System) clerk processes the migrant student 
records and assists students with securing needed social and medical services. 

Title I Migrant Objectives 

1. To comply with federal law requiring an annual evaluation of programs funded with 
Title I monies; and 

2. To provide information for decision makers on program effectiveness to facilitate 
decisions about program modifications. 

Scope and Method 

Quantitative and qualitative data will be gathered to identify the benefits of the Title 
I Migrant program to this special' needs population in AISD. A count of students served and 
demographic data will be reported to TEA and further student data will be pulled from the 
Public Education Information Management System (PEIMS). Federally mandated parental 
involvement component activities will be measured for effectiveness by parental 
attendance/participation. Also, during the 1998-99 school year, Title I Migrant evaluation 
staff will undertake a study of the tutoring program to determine the specific effects on the 
students receiving this migrant-funded service. 
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Title IV Safe and Drug-Free Schools and Communities Program 

Program Contact: Mary Walker 

Evaluation Staff: Martha Doolittle, Suzanne Sharkey 

Program Description 

Since the 1 987-88 school year, the Austin Independent School District (AISD) has 
received funding through the federal Safe and Drug-Free Schools and Communities (SDFSC) 
grant, a component of Title IV. This funding originated in 1986 under Public Law 99-570 as 
the Drug-Free Schools program, later amended as the Drug-Free Schools and Communities 
program, and in 1994 was again broadened under Public Law 103-382 to become known as 
the Safe and Drug-Free Schools and Communities program. The expressed function of the 
SDFSC grant monies is to supplement, but not supplant, local school districts' efforts toward 
drug abuse education and prevention and to promote a safe, violence-free school 
environment. 

The 1998-99 AISD Title IV application, including 1997-98 roll-forward and 
supplemental funds, totals $1,112,635. In addition to its formula allotment ($405,830) and 
roll-forward funds, the district received funds through a Title IV supplemental grant provision 
(P.L. 103-382, §41 13(d)(1)). Under this provision, the Texas Education Agency apportions 
30% of the state’s Title IV funds to not more than 10% of Local Education Agencies (LEAs) 
demonstrating greatest need, based on districts’ annual evaluation reports to the state. In the 
1997-98 school year, the district was awarded a supplement of $525,976 through this 
competitive process. The district has asked TEA permission to roll forward these 
supplemental funds into FY 1998-99. 

With these funds, the district is undertaking an effort to review and revise its drug and 
violence prevention and education efforts. In the past, the district’s SDFSC program has 
been somewhat fragmented in its approach toward substance use prevention education. For 
the past two years, however, the district has begun taking steps to develop and implement a 
comprehensive, integrated drug and violence-prevention curriculum that includes adaptations 
of proven research-based programs and practices. This effort is in accordance with recent 
federal directives: (1) to align local school district’s SDFSC programs with measurable goals 
and objectives for drug and violence prevention, and (2) to use evaluation as a method of 
strengthening those programs. 

Thus, evaluation of the 1998-99 SDFSC program will include assessments in the 
following areas: (1) development, piloting, and implementation of curriculum integration; (2) 
programs and activities that are initiated or expanded through supplemental funding; and (3) 
established AISD SDFSC-sponsored programs. 

Campus-Based Programs 

Since the 1993-94 school year, SDFSC funds have been available to all campuses to 
allow school staff the latitude to create innovative programs toward a drug-free learning 
environment. Each campus that applies for these funds must develop a proposal that 
articulates the linkage between the school's planned expenditure, their campus improvement 
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plan, and the goals and objectives of the federal SDFSC legislation. Campuses are encouraged 
to leverage their funds by combining resources and programs across campuses or within 
vertical teams. In the 1997-98 school year, campus-based programs were funded at a level of 
approximately $1.10 per pupil. It is anticipated that funding will increase to $2.00 per pupil 
for the 1998-99 school year. 

Private Schools and Neglected or Delinquent Facilities 

By law, private schools and neglected or delinquent facilities within the district's 
boundaries are offered the opportunity to receive SDFSC funds for the development or 
expansion of comprehensive (pre-K through grade 12), age-appropriate programs related to 
safety and to prevention of the abuse of tobacco, alcohol, or illegal substances. Funds may be 
used for acquisition or implementation of programs, staff development, consultants, 
materials, supplies, equipment, and registration fees for workshops or training. In the 1997- 
98 school year, 15 private schools and one neglected or delinquent facility received SDFSC 
fund allotments, and an estimated 1 8 private schools and seven neglected or delinquent 
facilities will participate in the SDFSC program in 1998-99. 

Pre-K-12 Curriculum Support 

The objectives of the curriculum support component of the grant are threefold: 

1. To design, develop and implement age-appropriate curricula for students in grades 
Pre-K through 12 that cover areas of drug and alcohol education and prevention; 

2. To provide in-service training to teachers and counselors on how to make the best 
use of materials and consultants dealing with drug and alcohol education and 
prevention; and 

3. To provide resources for administrators, counselors, and teachers to attend 
conferences and stay current with alcohol and drug education and prevention 
programs. 

This year is a continuation of a multiyear, comprehensive review and modification of 
the district’s drug, alcohol and violence prevention curriculum resources. A review of student 
competencies from the Guidance, Health, Science, Language Arts and Physical Education 
curricula will continue with the identification of strands related to the prevention of drug 
abuse and violence. In order to integrate SDFSC-related curriculum materials into the 
district's core curricula, staff will research, modify, purchase and distribute materials to 
support the development of those competencies. This step has been achieved in the 
Guidance and Counseling curriculum, and is in process for the Health - Science curriculum. 
Teachers and counselors will be trained and be responsible for the delivery of curriculum. 

Peer Assistance and Leadership (PAL) 

PAL is a peer-helping program offering course credit to selected eighth, eleventh, and 
twelfth graders who function as peer helpers ("PALs") to other students ("PALees") at their 
own schools as well as at feeder schools. The grant provides for a half-time consultant who 
serves as the district's program coordinator and consultants to provide additional training, 
curriculum support, and student conferences. 
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ROPES Retreat Program (ROPES) 

The ROPES Program (Reality Oriented Physical Experiential Sessions) is operated by 
AISD's Office of School-Community Services. The ROPES program, a retreat workshop 
designed to serve both AISD students and staff, is a series of teambuilding exercises revolving 
around a set of physical challenges. There are experiential education activities designed to 
develop such skills as team building, trust, communication, decision making, problem solving, 
and resistance to negative peer pressure. The grant provides the salary for the project 
facilitator and two program assistants, as well as funding for facility rental, transportation 
costs, and substitutes to allow participation by teachers. 

Student Assistance Program (SAP) 

The SAP is a school-based process aimed at helping students address difficulties that 
affect their ability to perform successfully in school. In 1997-98, this program focused on 
student problems such as academic difficulties, family/social issues, and misconduct. The SAP 
provides training to elementary and secondary staff to enable them to recognize and assist 
students who are experiencing personal and academic difficulties. 

Alternative Education Programs 

In addition to the programs discussed above, a portion of the 1998-99 SDFSC grant 
will be used to support some of the district's alternative education programs. The Alternative 
Learning Center (ALC) serves all secondary campuses in the district through transition, 
consultation, and direct services to referred students. The programs at the ALC were 
recently refocused on improving students' behavior and imparting fundamental skills 
necessary for academic and social success. These programs are intended to help students 
achieve success when they return to their home campuses. Toward these ends, SDFSC funds 
will be used to support the following activities and positions: 

SUPER I Program. SUPER I is designed to serve secondary school students who 
have committed a first-time, non distribution alcohol- or drug-related offense. This 
voluntary program, which requires participation by both the student and parent (or other 
significant adult figure), consists of four two-hour sessions focusing on the physical, social, 
and legal consequences of substance use and places a strong emphasis on family 
communication skills. The incentive for participation is to allow the student to return to his 
or her home campus after two weeks rather than the more usual six weeks. The SUPER I 
program is co supported by a corporate sponsor, and additional cycles of the program are 
funded on an as-needed basis by the SDFSC grant. 

Positive Families. In addition to the training provided to parents by the SUPER I 
program, the Positive Families program offers training workshops to parents and students on 
anger management, communication, conflict resolution and problem solving strategies. The 
grant provides for one parent training specialist, and will also provide for general supplies, 
refreshments, and support for the program. District facilitators will be trained to offer the 
workshops. The workshops can be modified to meet the needs of those families attending. 

Other Program Support 

In addition to the programs and positions discussed above, the grant will also pay, at 
varying levels, a portion of salary for the following positions: SDFSC Program Facilitator, 
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five Community Specialists, five Instructional Coordinators, Secretary/Accounting Clerk, 
AISD Campus Police Officer, Evaluation Analyst, Evaluation Associate. 

Evaluation Objectives 

1 . To comply with federal law requiring an annual evaluation of programs funded with 
Title IV monies (P.L. 103-382, §4117 (b)(1)(A) and §4117 (b)), and as required under 
TEC §29.081; and 

2. To provide information for decision-makers on program effectiveness to facilitate 
decisions about program modification. 

Scope and Method 

GENeric Evaluation SYStem (GENESYS) 

The effectiveness of individual SDFSC-supported programs in influencing variables 
such as student achievement (as measured by course grades and test scores), attendance rates, 
retention rates, discipline rates, and so on for specified populations will be measured using 
OPE’s GENESYS program. For more detailed information regarding GENESYS, see 
GENESYS 1990-91: Selected Program Evaluations (Publication 90.39). 

Student Survey 

The Drug-Free Schools Act of 1986 requires that agencies receiving funds describe the 
extent of the current alcohol and drug problem in the schools. The National Commission on 
Drug-Free Schools Final Report (1991) recommends a survey to assess these problems. As a 
recipient of SDFSC monies, AISD is obliged to collect and report this information. Two 
surveys are used in alternating years: Texas School Survey of Drug and Alcohol Use, a 
statewide survey, and Student Substance Use and Safety Survey (SSUSS), a locally produced 
survey that duplicates many of the items from the Texas School Survey. This year, the 
Student Substance Use and Safety Survey (SSUSS) will be administered to assess levels of 
student self-reported substance use, attitudes about substance use, and opinions of student 
violence and school safety. 

Staff Survey 

Staff opinions of the presence of alcohol and drugs on campuses and safety concerns 
are assessed in an annual survey to a random sample of staff (teachers and administrators). In 
addition, questions are included on staff familiarity with and training in SDFSC-relevant 
curricula or topics. 

Campus Expenditure Proposal and Year-end Report 

In order to qualify for SDFSC funds, at the beginning of the school year, each campus 
will be required to submit a proposal outlining planned expenditures. Each proposal must 
include an explanation of how the campus’ program(s) will advance the goals and objectives 
of the SDFSC legislation, and how this supports campus improvement plans. Campuses will 
be required to submit a year-end report on their programs which will include information such 
as the number of staff and students served directly and indirectly by the programs, how funds 
were spent, and the extent to which the goals and objectives of SDFSC legislation were 
addressed. This information is required for reporting to TEA. 
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Private School/Neglected or Delinquent Facility Year-end Report 

Similar to the campus-based programs, the private schools and neglected or delinquent 
facilities will be asked to report all SDFSC-related expenditures as well as the number of 
students and staff served through those expenditures and to describe the programs and 
activities sponsored by the grant. This information is also required for reporting to the TEA. 

Pre-K through 12 Curriculum Support 

TEA’s (1997) publication of the Texas Prevention Curriculum Guide: Drug and 
Violence Education (DAVE) is the drug, alcohol, and violence prevention education 
curriculum document that will be integrated into different academic curriculum areas over the 
next several years. AISD's Guidance and Counseling curriculum guide has been updated and 
will be implemented during the 1998-99 school year to all AISD campuses. The district’s 
Health curriculum, Making Healthy Choices (1992), is in the process of being updated and will 
include the DAVE curriculum materials. Evaluation will require cooperation with staff from 
the different curriculum areas in order to monitor the development of curriculum materials, 
teacher training, and the pilot year of teaching the curricula to students. 

PAL 

The PAL program will be evaluated using a combination of assessment techniques 
including interviews with PAL program participants and the PAL coordinator, a content 
analysis of PAL/PALee activities, and follow-up of last year’s participants. 

ROPES 

Evaluation of the ROPES Program will include interviews with the program 
coordinators, surveys of staff and student participants, and follow-up of previous year’s 
participants. 

SAP 

Evaluation of SAP staff training will consist of an examination of program records 
related to staff training, follow-up with staff members that have been trained in previous 
years, and an interview with the SAP coordinator. 

Alternative Education Programs 

Much of the restructuring effort in the district’s SDFSC programs focuses on 
programs for students considered at greatest risk for substance use and other disciplinary (e.g., 
class disruption, fighting) problems. Many of the students referred to the district’s 
alternative education facilities fit this definition. Appropriate evaluation instruments will be 
designed to reflect the development and implementation of these programs. For instance, 
current program participants’ behaviors and opinions will be evaluated, and follow-up 
measures will be collected to assess the long-term impact of the programs (e.g., SUPER I, 
Positive Families) on students academic and behavioral progress. 

Other Program Support 

Staff interviews and surveys will be used to assess the roles of SDFSC support staff in 
the implementation of SDFSC programs in the district. 
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Title VI Program 



Program Director: 
Evaluation Supervisor: 
Evaluation Staff: 



Terry Ross 
Gloria Zyskowski 
Ertha Patrick 



Program Description 

Title VI provides federal funds to states through the Elementary and Secondary 
Education Act of 1965 as amended by the Improving America’s Schools Act of 1994 (Public 
Law 103-382). States receive Title VI funds based on their school-age population and then 
allocate at least 80% of these funds to local districts based on enrollment. Title VI can 
support programs or services in the following target assistance areas: school reform activities 
that are consistent with the Goals 2000: Educate America Act', instructional and educational 
materials; programs to provide for the educational needs of gifted and talented children; 
school improvement programs or activities; technology related to the implementation of 
school-based reform programs; activities to improve higher order thinking skills of 
disadvantaged elementary and secondary school students; and programs to combat illiteracy 
in the student and adult populations. Local education agencies are encouraged to maximize 
the effectiveness of Title VI funds by coordinating with other federal and state programs to 
provide an integrated, coherent delivery of services to increase student achievement. AISD 
programs funded by Title VI for the 1998-99 school year are described below. 

Reading Recovery 

Reading Recovery is a supplementary reading program for grade 1 students, who are 
identified for possible participation based on beginning-of-the-year teacher rankings of 
students in their classrooms. Students in the bottom one-third of the rankings are then 
selected for Reading Recovery services through additional teacher assessment. The goal of 
the program is for students to exit the program and return to their classroom at the average 
reading level of the class. 

Private Schools and Neglected or Delinquent Facilities 

By law, Title VI funds are available through AISD to private, nonprofit schools and 
neglected or delinquent Facilities within AISD boundaries. Private nonprofit schools and 
neglected or delinquent Facilities are contacted in the spring to determine if they would like 
to participate in Title VI programs during the upcoming school year. Applicants must meet a 
number of eligibility requirements related to compliance with federal nondiscrimination laws 
and nonprofit status. Funds are then allocated to approved applicants on a per-pupil basis for 
purchase of items selected by schools or facilities. All purchases are made through AISD, and 
the district retains title to, and exercises administrative control of, all equipment and 
supplies. Equipment and supplies that are placed in a private school must be used for Title 
VI purposes only, and must be able to be removed easily should a school or facility become 
ineligible for continued funding. 
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Library Resources 

Title VI allocates funds for each elementary, middle/junior high, and high school to 
purchase library resources for the purpose of improving the quality of instruction. Title VI 
funds may be used to purchase supplementary library materials, assessments, reference 
materials, media materials, and instructional/educational materials for any subject area and for 
any grade level. However, materials purchased through Title VI must focus on a particular 
purpose, reflect high academic standards, and must be part of an overall education reform 
program. Purchased materials are labeled as Title VI to indicate their funding source. 

Visiting Teachers 

As part of the Department of School Support for AISD, the mission of the visiting 
teacher program is to support campuses in achieving success for all students by connecting 
district and community resources and providing direct and indirect services to students, 
parents, and staff. Visiting teachers provided by AISD are required to have a master’s degree 
and certification in teaching, counseling, or social work. In addition, visiting teachers are 
required to have a minimum of three years' experience in classroom teaching or social work. 
Visiting teachers are also required to have a thorough knowledge of AISD policies and 
academic requirements; knowledge of social services' and programs available to meet a range 
of health, income, and social services needs; as well as information about state, local, and 
federal laws affecting the lives of students and families. 

During the 1998-99 school year, Title VI will fund two visiting teachers to provide 
support services to approximately 12 AISD schools. 

Volunteer Coordinator 

During the 1998-99 school year, a half-time Coordinator of Volunteers will provide 
support for campus recruitment, retention, and recognition of community volunteers. The 
duties of this position include development and distribution of materials, volunteer 
recruitment, maintaining a central database of volunteer hours, establishing ongoing 
communication with other area volunteer agencies, and working with community 
organizations to support campus needs. 

Summer School Tuition 

During the 1998-99 school year Title VI will provide funding for AISD high school 
students to attend summer school at Huston-Tillotson College. 

Mathematics Training Specialist 

During the 1998-99 school year, Title VI will fund a mathematics training specialist 
to conduct district-wide training workshops on a variety of topics for AISD mathematics 
teachers. The mathematics specialist will also provide continued assistance to teachers who 
need additional help after completing a workshop. The mathematics specialist will help plan, 
teach, and model workshop strategies for AISD teachers. 

Science And Health Workshop 

During the 1998-99 school year, Title VI will fund science and health workshops for 
AISD teachers. Workshops will be presented by the health and science curriculum team 
leader on a variety of topics. 
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Evaluation Objectives 

1. To comply with federal law requiring an annual evaluation of programs funded with 
Title VI monies; and 

2. To provide information for decision makers on program effectiveness to facilitate 
decisions about program modification. 

Scope And Method 

Evaluation of Title VI programs will be limited to a feedback report using data 
gathered by the Title VI evaluation associate and required by the Texas Education Agency 
(TEA). 
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Title VII-A Dual Language Project 



Program Director: 
Evaluation Supervisor: 
Evaluation Staff: 



Della Moore 
Ralph Smith 
Rosa Maria Gonzalez 



Program description 

In the 1995-96 school year, AISD received a five-year grant through Title VH-Part 
A to implement a program of Developmental Bilingual Education (DBE) at two elementary 
schools, Metz and Sanchez. Both schools have a large percentage (over 25%) of limited- 
English proficient (LEP) students. 

The DBE program requires dual language (two-way) instruction in English and a 
second language (in this case, Spanish) while mastering subject matter skills and meeting grade 
promotion requirements. The DBE contrasts with the Transitional Bilingual Education 
(TBE) programs, which require instruction in English and the non-English native language, 
with a gradual transition to English-only instruction. 

The DBE program funded in AISD is titled “The Dual Language Project (DLP): Dos 
Idiomas, Un Mundo.” The primary goals of the Dual Language Project are the following: 

1. Develop student oral proficiencies in English and Spanish; 

2. Develop student grade-level appropriate literacy in English and Spanish; 

3. Increase student academic achievement levels in reading, language, and mathematics; 
and 

4. Change the attitudes toward bilingualism of the students, parents, community, 
and staff. 

Evaluation Objectives 

1. To comply with federal law requiring an annual evaluation of programs funded with 
Title VII monies; and 

2. To provide information for decisionmakers on program effectiveness to facilitate 
decisions about program modification. 

Scope and Method 

The grant awarded to AISD by the U.S. Department of Education specifies that an 
independent evaluation be conducted to determine the effectiveness of the district’s Title VII 
DLP. OPE will employ a contractor, under the supervision of an evaluation associate and an 
evaluator, to carry out the evaluation. In the fourth year of funding, the evaluation will 
continue to collect and analyze data from a number of sources, including the following: 

1. The Student Master File will provide information on the students’ grade level, age, 
sex, ethnicity, school history, and parental income; 

2. The LEP file will provide information on the LEP status, home language, language 
dominance, and program entry date; 

3. The DLP-Language Assessment Scales (LAS) file will provide information on the 
LAS levels Oral Proficiency Test, both in English and in Spanish; 
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4. The ITBS file provides information on the grade equivalent, national percentile rank 
score, and the number of correct items on the reading, language, and mathematics 
ITBS skill batteries given to the DLP students; 

5. OPE’s GENESYS file will supply demographic and achievement information on the 
DLP students to determine programmatic progress; 

6. The Project Specialist will be interviewed to assess project management and 
implementation of the DLP; 

7. A survey of professional staff will be conducted to assess the staff academic 
preparation and expertise in the teaching methodology for the DLP; 

8. A revised version of the Bilingual Attitudinal Questionnaire will be administered to 
parents to assess their perceptions about participation in the DLP, and about being 
bilingual; and 

9. The biennial report mandated by federal law will be submitted to the Office of 
Bilingual Education and Minority Language Affairs (OBEMLA) in December 1999. 
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TITLE VII-A Rethinking Education for Minority Students 

(REFORMS) 



Program Director: 
Evaluation Supervisor: 
Evaluation Staff: 



Della Moore 
Ralph Smith 
Rosa Maria Gonzalez 



Program Description 

The project, currently in its second year, is intended to enhance and expand the 
locally funded ESL/ Transitional Bilingual Education Program in place at the Fulmore Middle 
School in Austin Independent School District. The project will also strengthen current and 
future reform and restructuring of the school relative to addressing the needs of the limited 
English proficient (LEP) students. The project builds on the Carnegie Foundation-supported 
reform efforts already begun in the school and will shift the instructional program for LEP 
students to a schoolwide program in which all instructional personnel will be responsible for 
and will be better prepared to provide effective instruction to LEP students and address issues 
of language and cultural diversity. 

Since 67.5% of the school's population is Hispanic, with over 20% LEP, the shift to 
a schoolwide ESL/Bilingual Education program will take a three-pronged approach: 

1. Professional Development. A five-tiered model will incorporate state-of-the-art 
delivery strategies including study groups, action research, structured 
observation/reflection, networking, and participation in professional conferences. 

2. Materials Acquisition. Supplementary resources will be purchased in all content 
areas to ensure that LEP students can participate in the school's interdisciplinary 
curriculum. 

3. Parent Education. The adult/parent ESL classes will be expanded to accommodate 
parents currently on the school's waiting list. 

This grant proposal supports the National Goals 2000 by a) promoting literacy in 
young adults, b) increasing students' competency in core subjects, and c) improving the 
quality of staff professional development. This grant also complements Title I, special 
education, and gifted and talented programs in the district. 

Evaluation Objectives 

1 . To comply with federal law requiring an annual evaluation of programs funded with 
Federal monies; and 

2. To provide information for decision makers on program effectiveness to facilitate 
decisions about program modification. 

Scope and Method 

The Office of Program Evaluation will employ an independent evaluation consultant 
who, under the supervision of an evaluation associate and an evaluator, will carry out the 
evaluation. In the second year of funding, the evaluation will continue to collect and analyze 
data from a number of sources, including the following: 
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1. The Student Master File will provide information on the students’ grade level, age, 
sex, ethnicity, school history, and parental income; 

2. The LEP file will provide information on the LEP status, home language, language 
dominance, and program entry date; 

3. The ITBS file provides information on the grade equivalent, national percentile rank 
score, and the number of correct items on the reading, language, and mathematics 
ITBS skill batteries given to the DLP students; 

4. OPE’s GENESYS file will supply demographic and achievement information on the 
LEP students to determine programmatic progress; 

5. The Project Specialist will be interviewed to assess project management and 
implementation to the REFORMS grant; 

6. A survey will be administered to teachers who participated in the project’s training 
component; 

7. Other performance and context indicators will be examined to determine program 
effectiveness; and 

8. The final report mandated by federal law will be submitted to OBEMLA in 
October/December 1999. 



ERIC 



29 



27 



98.01 



Agenda 1998-99 



Title VII-C Emergency Immigration Education Program 

Program Director: Della Moore 

Evaluation Supervisor: Ralph Smith 

Evaluation Staff: Rosa Maria Gonzalez 

Program Description 

Title VII, Part C, incorporating the former Emergency Immigration Education 
Program, provides grant funds "...to assist local education agencies that experience 
unexpectedly large increases in their student population due to immigration to (1) provide 
high-quality instruction to immigrant - children and youth; and (2) help such children and 
youth (a) with their transition into American society; and (b) meet the same challenging 
state performance standards expected of all children and youth." 

Evaluation Objectives 

1. To comply with federal law requiring an annual evaluation of programs funded with 
Title VII monies; and 

2. To provide information for decisionmakers on program effectiveness to facilitate 
decisions about program modification. 

Scope and Method 

The focus of the evaluation on the immigrant program is currently being determined. 
The data gathered will be reviewed to determine whether the program contributes to 
providing high-quality instruction and assists immigrant students in meeting state 
performance standards. Data for the 1998-99 evaluation will be secured from the following 
sources: 

1 . Information concerning program services and expenditures will be collected from 
program staff; 

2. Files and rosters on immigrant students will be created to gather information from 
other centralized files; and 

3. Prior-year information about immigrant students will be obtained from archived 
computer files and evaluation data files. 
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1998-99 Evaluation Overviews for Locally and Other Federally 

Funded Programs 
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Academics 2000: First Things First 

Program Administrator: Susan Kemp 

Evaluation Staff: Holly Koehler, Holly Williams 

Program Description 

Academics 2000 is the Texas initiative funded under Public Law 103-227, the Goals 
2000: Educate America Act. The primary objective of the Academics 2000 local grant 
program is to promote reading proficiency for students in prekindergarten through fourth 
grade. In order to meet this primary objective, programs are developed that focus on the 
planning and implementation of effective, research-based initiatives to improve reading 
proficiency, including intensive and sustained professional development that encompasses 
effective instruction strategies and methodologies. Some programs also include preservice 
education activities. 

Last year, AISD received eight Academics 2000 grants, involving 1 7 elementary 
school campuses. Six new grant applications have been submitted for the 1998-99 school 
year. 

Evaluation Objectives 

1. To comply with federal law requiring an annual evaluation of programs funded with 
Academics 2000 monies, and 

2. To provide information for decision makers on program effectiveness to facilitate 
decisions about program modification. 

Scope and Method 

Assistance in writing the evaluation section of grant applications will be provided to 
all campuses that apply for Academics 2000 funding. Demographic and achievement data 
will be provided to Academics 2000 recipients for use in completing end-of-the-year 
evaluation reports. In addition, recipients can request in-depth evaluations of all or part of 
their Academics 2000 programs and/or assistance in completing their end-of-the-year 
evaluation reports. 

So far, three Academics 2000 recipients have requested in-depth evaluations for the 
1998-99 school year and have budgeted for evaluation. (This number may increase as new 
recipients are notified of their acceptance.) The Reading One-To-One component of Harris 
Elementary School’s Academics 2000 program, “Early Parent Involvement to Promote 
Reading Project” will be evaluated using Woodcock-Johnson/Woodcock-Munoz pre- and 
post-test scores, TAAS scores, and Reading One-To-One program documentation. The 
Project READ portion of Blackshear, Govalle, Jordan, and Oak Springs/Rice Elementary 
School’s “Blackshear Project” will be evaluated using pre- and post-tests of staff 
development, teacher observations of student teachers, Analytical Reading Inventories, and 
TAAS scores. Finally, the Academics 2000 program at Cook/Wooldridge Elementary 
Schools, “The Quail Creek Family Reading Program,” will be evaluated using surveys of 
participants, parents, staff, and feedback from Parent Night. 
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Annual Analysis of TAAS Results 

Program Contact: Darlene Westbrook 

Evaluation Staff: Holly Williams 

Program description 

The Texas Assessment of Academic Skills (TAAS) is a state-mandated, criterion- 
referenced (CRT) or mastery test that has been administered since the 1 990-9 1 school year. 
TAAS measures student mastery of the statewide curriculum in reading and mathematics at 
grades 3 through 8 and at exit level, and in writing at grades 4 and 8 and at exit level. Other 
TAAS tests are administered, however, the results presented and reviewed in the TAAS report 
are limited to results that are included in the calculation of the State Accountability System 
Ratings (reading and mathematics at grades 3-8 & exit, and writing at grades 4, 8, & exit). 

Evaluation Objectives 

1. To analyze the impact of district initiatives and strategies for the prior year, and 
recommending initiatives and strategies for the coming school year. 

2. To provide TAAS information to district staff, and to provide an example of an 
analysis of TAAS results for campus staff to follow. 

Scope and method 

The TAAS Analysis Committee is charged with formulating an annual analysis of 
district TAAS results. The annual analysis includes identifying areas of progress and areas of 
focus for the district, analyzing the impact of district strategies and initiatives on school 
performance, and recommending strategies and initiatives for the coming school year. 

The task force, with assistance from other staff, analyzes the TAAS results in 
different ways to answer questions about student achievement. District data are summarized 
by percentage of campuses making gains in each subject, at each grade level, and for each 
disaggregated group (all students, African American, Hispanic, White, & economically 
disadvantaged). Student performance is analyzed by grade, by disaggregated group and by 
subject over a three-year period. State TAAS results are compared to district TAAS results. 
Campus data over time are presented, as well. 




34 



31 



98.01 



Agenda 1998-99 



Austin Collaborative for Mathematics Education Program 

Program Administrator: Laurie Mathis 

Evaluation Staff: Michelle L. Batchelder, Holly Williams 

Program Description 

The Austin Collaborative for Mathematics Education (ACME) is a Local Systemic 
Change (LSC) initiative to implement reforms in K-8 mathematics instruction district-wide. 
The National Science Foundation (NSF) has granted the district up to $5 million to train 
teachers and support their adoption of a reform curriculum. The program is a collaborative 
effort of the Austin Independent School District, the Charles A. Dana Center, and the 
University of Texas at Austin. 

All of the K-8 mathematics teachers in the district will participate in the program. 
The training is long-term and focuses on teachers as learners. In the first two years of the 
program, teachers participate in summer institutes and five days of follow-up training during 
the academic year. The third year consists of five days of follow-up training only. 

Throughout the program, teachers also receive coaching from mathematics specialists and 
peers as well as release time for planning and team-building. In addition, principals and 
school counselors participate in two days of summer training to understand program goals, 
plan strategies to support the teachers, and develop forums in which to inform parents about 
the district’s mathematics program. 

Implementation of the program began in the summer of 1997 with 5th and 6th grade 
teachers across the district. In the summer of 1998, 4th and 7th grade teachers entered the 
program, as well as teachers from pilot schools in which all grade levels participate in training 
simultaneously. Teachers at other grade levels district- wide will enter the program in 
subsequent years. 

Program administrators have adopted the Connected Mathematics Project (CMP) 
and Investigations in Numbers, Data, and Space as curriculum resources. The district is 
purchasing materials for each K-8 classroom in each school, and materials in Spanish for 
Bilingual Classrooms. 

Evaluation Objectives 

1. To comply with the National Science Foundation (NSF) requirement of an annual 
evaluation of programs funded with NSF monies; 

2. To provide administrators information about program implementation and 
effectiveness to facilitate decisions about program modification. 

Scope and Method 

The evaluation of the ACME program has two components. The national 
component fulfills NSF funding requirements and provides information about the program 
consistent with other mathematics reform programs across the nation. The local component 
addresses the needs of district program administrators and facilitates the development of the 
program. The evaluator collects and analyzes both qualitative and quantitative data. Rose 
Asera, Ph.D., Coordinator of Research and Evaluation at the Dana Center, and Holly 
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Williams, Ph.D., Assistant Director of the Office of Program Evaluation for AISD jointly 

supervise local evaluation. 

NSF has contracted Horizons Research, Inc. to design and direct the national 

component of the evaluation, which is the same for all of the LSC initiatives that receive 

NSF funding. To fulfill the Core Evaluation of the national component, the evaluator: 

1 . Conducts a formative evaluation of professional development activities during the 
summer and academic year, including two to three observations the first year and five 
to eight observations in subsequent years; 

2. Conducts a formative evaluation of the implementation of the curriculum in 
classrooms, 1 0 observations per year; 

3. Gathers survey responses from a sample of K-8 mathematics teachers and all of the 
elementary and middle school principals in the district; 

4. Interviews several mathematics teachers every year after the baseline year; and 

5. Collaborates with AISD and the Dana Center evaluation supervisors as well as 
program staff to design and implement the summative evaluation, which will include 
qualitative and quantitative data. 

The local component of the evaluation includes the following plans: 

1 . To document the vision and history of the program, the evaluator will interview 
district administrators and program staff. 

2. To determine why 17% of 5 th and 6 th grade teachers did not attend at least 20 hours 
of training in the first year of the program, a survey was distributed to these teachers 
in the spring of 1998. 

3. To examine the quality, content and atmosphere of professional development 
training, the evaluator will observe sessions and provide feedback to program staff in 
group form. 

4. To evaluate teacher attitudes and experiences in professional development, the 
evaluator will participate in professional development training and interview teachers 
informally about their perceptions. 

5. To describe the strengths and weaknesses of district-wide implementation of a reform 
curriculum, the evaluator will study the evolution of the program in depth at four 
schools in the 1998-1999 academic year. This part of the evaluation will investigate 
changes in mathematics curriculum and instruction in classrooms, teacher perceptions 
of constraints and supports for implementation, networks of communication and 
interaction that teachers use in adopting the curriculum, and the role of campus 
administrators in implementation of the curriculum. 

6. To evaluate the effects of the reform curriculum and instruction on student 
mathematics achievement, TAAS and ITBS scores will be analyzed for the district by 
campus, and by disaggregated groups, such as low socioeconomic status, ethnicity, and 
English proficiency status. 
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Bilingual/ESL Programs 

Program Director: Della Moore 

Evaluation Supervisor: Ralph Smith 

Evaluation Staff: Rosa Maria Gonzalez 

Program Description 

Texas law requires that all students with a Language Other Than English (LOTE) be 
assessed to determine their level of English proficiency. Those identified as limited-English 
proficient (LEP) must be provided one of two basic programs: 

1 . Bilingual Education (BE), a transitional program of dual-language instruction 
including instruction in the home language and English as a Second Language (ESL) 
provided to students in any language classification for which there are 20 or more 
students enrolled in the same grade level; and 

2. English as a Second Language (ESL), a program of specialized instruction in English 
provided to students not receiving bilingual education and to students whose parents 
refuse dual-language instruction. 

Parental permission is required for either program. Some LEP students are served only by 
special education. 

Evaluation Objectives 

1 . To comply with state law requiring an annual performance evaluation of the district's 
Bilingual/ESL Programs; and 

2. To provide information for decisionmakers on program effectiveness to facilitate 
decisions about program modification. 

Scope and Method 

The focus of the evaluation will be the impact of the district's Bilingual/ESL 
Programs on the academic progress, in either language of instruction, of its LEP students, and 
the extent to which they are becoming proficient in English. The evaluation will document 
the number of teachers and teacher assistants trained, the frequency and scope of the 
training, and the results of the professional training activities. Data from the 1998-99 
school year and longitudinal information from previous years will be included in the 
evaluation. Data for the 1998-99 evaluation will be secured from the following sources: 

1 . The LEP Master File contains a wide range of information about each LEP student, 
including the student's LEP status, program, home language, language dominance, 
program service dates, and performance on standardized achievement tests. 

2. Both criterion-referenced (TAAS) and norm-referenced (ITBS and La Prueba de 
Realizacion) test score data will be used. 

3. The Student Master File (SMF) will provide basic information about student grade 
level, ethnicity, and low-income status. 

4. Information about program services and staff training will be obtained from program 
staff interviews and records. 
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5. Prior-year information concerning LEP students will be obtained from published 
evaluation reports and evaluation data files. 

The following analyses will be performed: 

1 . Indicators of LEP student performance - achievement, attendance, retention, school 
leaver rates, etc. - will be examined for trends over time. 

2. Performance indicators for served LEP students and LEP students whose parents 
refused services ("refusals") will be compared to determine if program students differ 
significantly from the nonprogram students. 

3. The academic progress of LEP students who have exited from the program will be 
examined on a longitudinal basis. 
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Carl D. Perkins, Middle School Pregnant and 
Parenting Program 

Program Administrator: Gloria Williams 

Evaluation Staff: Holly Koehler, Holly Williams 

Program Description 

The Carl D. Perkins Vocational and Applied Technology Education Act (P.L. 101- 
392) assists states to expand and improve their programs of vocational education and provide 
equal access in vocational education to special populations. As amended in 1990, the Perkins 
Act focuses on increasing the participation of special populations, and on improving the 
quality of vocational programs in schools with the greatest need for improvement. 

The AISD Single Parents Project is funded through Title II, Subpart 1 of the Perkins 
Act: Programs to Provide Single Parents, Displaced Homemakers, and Single Pregnant 
Women With Marketable Skills and to Promote the Elimination of Sex Bias. The AISD 
Single Parents Project was designed to connect pregnant middle school students with social 
services and other resources, provide them with information, encouragement, and support, 
and encourage them to stay in school. 

This year, the AISD Single Parents Project will include emergency child care and in- 
home schooling to pregnant middle school students and new mothers. As in previous years, a 
social worker will be available for the students, although, this year, funding for the social 
worker position will be provided through another source. 

Evaluation Objectives 

1. To comply with federal law requiring an annual evaluation of programs funded with 
Carl D. Perkins Act monies, and 

2. To provide information for decisionmakers on program effectiveness to facilitate 
decisions about program modification. 

Scope and Method 

The focus of the evaluation will be on the satisfaction of the students served by the 
program. Qualitative data will be collected and analyzed to explore the effects of the 
program. Students will be surveyed and/or interviewed regarding how helpful they felt their 
participation in the program was and the nature of the services they received. In addition, 
program records and other data will be summarized for inclusion in a TEA year-end report. 
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Diversified Education through Leadership, 
Technology, & Academics (DELTA), 1998-99 

Program Administrator: Mollie Guion 

Evaluation Staff: Ralph Smith, Karen Wendt 

Program Description 

The DELTA program, an open-entry, open- exit, alternative diploma program, is in 
its fourth year of implementation in AISD. This competency-based dropout prevention and 
recovery program employs individualized and self-paced instruction and use of the NovaNET 
computer system to deliver district curriculum and to assist students in earning credits and 
passing the TAAS exam. 

Evaluation Objectives 

1. To provide information for decision makers on program effectiveness to facilitate 
decisions about program modification; and 

2. To document the extent and effect of implementation of the DELTA and 
Community-Based Alternative program in AISD. 

Scope and Method 

As in the previous three years, evaluation will focus on gathering data concerning 
students served, the number of former dropouts recovered, the number of at-risk students 
served, the number of credits earned, course credits assisted by NovaNet, the number of high 
school diplomas awarded to students, the percentage of students passing TAAS, and 
demographic information on students served. The collected data will be reviewed and 
analyzed to investigate the effectiveness of this competency-based diploma program for 
recovered dropouts and at-risk students. The program evaluation staff will collaborate with 
DELTA program staff to gather data on the effectiveness of the program in achieving its 
goals. Data gathered for the 1998-99 school year will be secured from the following sources: 

1. Information about program services will be collected from the program staff. 

2. Data will be collected every six weeks and given to program staff on disk. 

3. The student master file will provide basic information about student grade level, 
ethnicity, gender, special education, and limited-English-proficiency (LEP) status. 

4. TAAS results will be obtained from the TAAS file. 

5. Data collected during 1996-97 and 1997-98 will provide longitudinal information on 
student progress. 

If time permits, evaluation staff will also seek to supplement the evaluation by adding 
certain qualitative information, possibly including surveys and interviews with student 
participants, DELTA staff, and campus administrators. The evaluation may also include 
financial information, including program costs; for example, number of students enrolled in a 
co-op program, cost-per-student served, cost-per-credit earned, and cost-per-DELTA 
graduate. If feasible, the evaluation may also attempt to determine outcomes for former 
DELTA participants, graduates and non-completers both, using SOIC data 
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Excel Through Innovation 



Program Administrator: 
Evaluation Staff: 



Linda Bayley 

Holly Koehler, Holly Williams 



Program Description 

ExceL Through Innovation (ExceL) is a local funding initiative designed to improve 
student achievement at the elementary level through campus-based decision-making. ExceL 
funds provide a means for campuses to develop programs that meet the specific needs of the 
populations they serve. ExceL is a four-year project; the 1998-99 school year will be the 
third year that campuses receive ExceL funding. Award amounts to campuses are the same 
each year (with the exception of year one when an additional allotment for “capital outlay” 
was included). ExceL award amounts were based on the number of students not meeting 
minimum expectations on TAAS at each campus in the spring of 1995. In addition to the 
basic award amounts, each campus is given six days of staff development per year for each 
professional staff member. The original grant proposals at each campus were created in the 
spring of 1996 with staff, parent, and community participation. Since that time, campuses 
have been free to make adjustments to their programs, as needed, with the approval of their 
area superintendents. 

Evaluation Objectives 

1. To provide information for decisionmakers on program effectiveness to facilitate 

decisions about program modification. 

Scope and Method 

Each campus receiving ExceL funding is required to set yearly goals including at least 
one TAAS-related achievement goal. The 1998-99 ExceL evaluation will focus on 
achievement of these goals. TAAS data will be presented to address each campus’ TAAS- 
related achievement goals, and data provided by each campus will be presented to address 
achievement of additional campus goals. In addition, the evaluation will include detailed 
descriptions of campus programs and staff development activities. Budget information will be 
presented regarding expenditures on the campus and the district level. Finally, several 
schools will be selected, through area superintendent nominations and analysis of 
achievement results, for inclusion in the Best Practices section of the evaluation report. Best 
Practices principals and teachers will be interviewed regarding the success of their programs, 
and the main characteristics of successful ExceL programs will be presented. 
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Families Communicate Program 

Program Contact: Jo Ann Farrell 

Evaluation Staff: Martha Doolittle, Suzanne Sharkey 

Program Description 

The Families Communicate program offers training workshops to parents and 
students on anger management, communication, conflict resolution and problem solving 
strategies. Based on the Positive Families curriculum and program offered at the district’s 
Alternative Learning Center, Families Communicate was piloted during 1997-98 at selected 
AISD campuses. The goal of the program is to intervene with students who are having some 
disciplinary problems at their home campus before it becomes necessary to remove them for 
more serious offenses. Through the series of lessons, students and parents are taught 
communication and problem-solving strategies and given the opportunity to practice their 
new skills. Trained facilitators from the campuses (e.g., primarily teachers and counselors) 
deliver the program. Plans are to expand the class offerings in 1998-99 to other campuses. 
Evaluation Objectives 

1. To assess program goals by following up on student participants from the pilot year 
and the current year; and 

2. To provide information for decisionmakers on program effectiveness to facilitate 
decisions about program modification. 

Scope and Method 

GENeric Evaluation SYStem (GENESYS). The effectiveness of the program in 
influencing variables such as student achievement (as measured by course grades and test 
scores), attendance rates, retention rates, discipline rates, etc. for program participants will 
be assessed using OPE’s GENESYS program. For more detailed information regarding 
GENESYS, see GENESYS 1990-91: Selected Program Evaluations (Publication 90.39). 

Participant Surveys. Surveys will be used to identify problems that students and 
parents want to address during the class sessions, and to investigate student and parent 
opinions on what they have gained from the sessions. 

Program Staff Assessment. Program staff will be interviewed and surveyed about 
their opinions on whether the program goals are met, opinions about the class materials and 
program operation, and any concerns and recommendations for the program. 
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Professional Development: Literacy and Mathematics at Read 



Program Description 

The Professional Development Academy (PDA) provides professional development 
opportunities designed to enhance the knowledge, skills, and performance of all AISD 
employees. This evaluation will focus on training provided by PDA to support two 
districtwide initiatives: the new mathematics curriculum, specifically Investigations at the 
elementary level and the Connected Mathematics Project (CMP) at the middle school level; 
and literacy activities, specifically the use of the PALM (Primary Assessment of Language 
arts and Mathematics) at grades preK-2. However, because aspects of PALM are being 
refined to allow districtwide collection of student achievement data, the PALM professional 
development evaluation will be a formative evaluation designed to assist program staff in 
program improvement. 

The evaluation will be a continuation of efforts that were undertaken during the 
1997-98 school year. Initial evaluation activities at that time included establishing a database 
at PDA to track teachers who had received training in mathematics and literacy, and 
gathering qualitative information on teacher training provided at PDA through evaluation 
forms completed at the end of each training session. 

Evaluation Objective 

1. To provide information for decisionmakers on effectiveness of staff development in 

Mathematics and Literacy to facilitate decisions about modification of AISD staff 

development. 

Scope and Method 

In summer 1997, a group of 5th grade teachers and a group of 6th grade teachers 
received professional development at PDA in the use of the new districtwide mathematics 
curriculum resources: Investigations at grade 5 and CMP at grade 6. These groups of 
teachers received a series of follow-up training during the 1997-98 school year to provide 
additional assistance in implementing the new curricular resources into their classrooms. 
Similarly, a group of teachers assigned to grades preK-2 received training at PDA to instruct 
them on the PALM as an assessment tool for use in their classrooms. Throughout the 1997- 
98 school year, follow-up instruction was provided to support teachers implementation of 
PALM. New groups of teachers are continually being introduced to professional training in 
the mathematics and literacy initiatives, but the group of teachers trained in summer 1997 
will form the basis for this evaluation designed to provide baseline information on the 
training project. 

With the assistance of Laurie Mathis, Administrative Supervisor for mathematics, 
and Jacquie Porter, primary education specialist, seven schools (four for mathematics and 
three for literacy) will be selected in which to conduct a Best Practices study. The purpose of 
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the study will be to address the effectiveness of the training that has been provided since 
summer 1997 to prepare teachers in the implementation of these specific districtwide 
initiatives in their classrooms. Campuses will be selected based on the degree to which they 
are implementing the training in their classrooms. Both qualitative and quantitative data will 
be gathered to obtain a measure of the effectiveness of the professional development 
provided by PDA in terms of how well the mathematics and literacy initiatives are being 
implemented in classrooms of teachers who have been receiving training since summer 1997. 

During the site visits, which are to be conducted in November 1998 through February 
1999, the evaluation staff will conduct classroom observations and will gather additional 
information through interviews with teachers and administrators. In addition, surveys will be 
designed to gather opinions about the training that was provided by PDA and how well that 
training prepared the teachers for Classroom implementation of the district initiatives. 

In addition to the qualitative information gathered through observations, surveys, and 
interviews, the evaluation team will investigate outcome measures that will provide 
meaningful quantitative information on the effects of teacher training on student 
achievement. End-of-course assessments associated with the mathematics curricula could be 
employed, along with TAAS scores and course grades. The types of scores generated by 
PALM will also be studied to determine their usefulness for baseline information. 

Longitudinal information gathered by PDA over the two-year period covered by this 
overall evaluation effort will be used to identify who participated in the training and to 
investigate the results of the evaluation forms that all participants are required to complete 
at the end of each training session. This information will be supplemented by a literature 
review of professional development practices, particularly in the areas of mathematics and 
literacy programs. Results of these various data gathering efforts will be presented in a report 
that will be distributed in September 1999. 
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Professional Development System for 
Teachers and Administrators 

Program Administrator: Paul Shooter 

Evaluation Staff: Holly Williams, Holly Koehler 

Description of Professional Development System 
for Teachers and for Administrators 

The Administrator Appraisal System was implemented in the Fall of 1997, and the 
newly developed Teacher Professional Development System (TPDS) will be implemented in 
the Fall of 1998. The feedback from the Administrator Appraisal System and from the 
TPDS will be included in the planning process for Professional Development in the district. 
The major goal for the Administrator Appraisal System is to support professional 
development that will increase academic achievement for all students in AISD. The major 
goals for the TPDS include the following: 

1. To foster ongoing professional development for AISD teachers, 

2. To increase educators' knowledge and skills, 

3. To create a professional atmosphere that ensures more collaboration and collegial 
conversations between teachers and administrators, 

4. To increase teachers’ perception of their classroom effectiveness, 

5. To enhance the delivery of quality educational services for all students, and 

6. To increase academic achievement for the diverse student population of AISD. 

Evaluation 

An evaluation is planned to investigate the effectiveness of the Administrative 
Appraisal System and the TPDS in reaching these goals. The evaluation will have three 
major components: a formative evaluation (based on qualitative data analysis and an 
investigation of reliability of the system); a summative evaluation at the end of the second 
year of implementation (measuring staff outcomes qualitatively and quantitatively; and an 
investigation of validity of the system); and an ongoing evaluation. After the formative 
evaluation is completed, the effectiveness of the Administrative Appraisal System and the 
TPDS in reaching the major outcome goals can be assessed in the summative evaluation. 
Qualitative and quantitative data will continue to be gathered annually for ongoing evaluation 
after the formative and summative evaluations are completed. 

Formative Evaluation Component 

A formative evaluation will be carried out during the first year of implementation to 
identify strengths or weaknesses of the of the Administrative Appraisal System and the 
TPDS and in order to gather information to be used in making any needed adjustments to the 
systems. The formative evaluation will include investigation of teacher and administrator 
attitudes. These data will be gathered through the employee coordinated survey. 



ERIC 



45 



42 



98.01 



Agenda 1998-99 



Main issues to be addressed by teachers will include the following: 

1. Teacher perception of the usefulness of the TPDS in determining their own 
professional development needs, 

2. Teacher perception of fairness of the new system, 

3. Teacher perception of whether the TPDS is effective in increasing collaboration 
among teachers and administrators on a campus, 

4. Teacher perception of increased effectiveness of classroom instruction, 

5. Teacher attitude towards the utility of the holistic scoring approach employed in the 
new system versus the analytical checklist used in the old system, and 

6. Teacher perception of the quality of training for the TPDS. 

Main issues to be addressed by administrators will include the following: 

1 . Administrator perception of usefulness of the Administrator Appraisal System and 
the TPDS in guiding decisions regarding professional development, 

2. Administrator perception of whether the TPDS is effective in increasing 
collaboration on a campus, 

3. Administrator perception of the ease of use of the Administrator Appraisal System 
and the TPDS, 

4. Administrator perception of increased effectiveness of classroom instruction, 

5. Administrator perception of the cost in time of the holistic scoring approach versus 
the benefit of additional information, and 

6. Administrator evaluation of training in the use of the TPDS. 

Summative Evaluation component 

Once formative evaluation determines that the Administrator Appraisal System and 
the TPDS are being used appropriately, a summative evaluation can be carried out and the 
external validity of the instruments can be investigated. Issues that will need to be addressed 
for evaluation include developing a methodology for storing performance assessment data in 
a mainframe database that can be accessed for use in the evaluation. 

Methodology planned for the external validity study for the TPDS entails correlating 
teacher assessment results with achievement results for students. The analysis will be 
performed on data at the district level; teachers will not be identified in the analysis. 

Average TLI gains for students in a teacher’s classroom will be used to measure achievement 
results; this student outcome number will be correlated with teacher appraisal results for all 
teachers/classrooms in the district. If it is true that Distinguished and Proficient teachers, as 
measured by the TPDS, lead to more positive student achievement outcomes, a linear trend 
in the data is expected. The number of grievances before and after implementation of the 
TPDS will be compared as well. Qualitative data gathered during the second year of 
implementation, through the district's employee coordinated survey will be included in the 
summative evaluation. At the end of the year a random sample of growth plans for the 
teacher and the administrator appraisal systems will be reviewed to check on follow through 
of planned professional development. 
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The information gathered through the formative evaluation to be used for system 
improvement will be presented to administrative staff during the 1998-99 school year. The 
formative and the summative evaluation results will be presented in a summary report to be 
produced by the AISD Office of Program Evaluation at the end of the second year, 1999- 
2000. The qualitative data for the ongoing evaluation will be collected annually after the 
second year of implementation and will be presented to administrative staff so that the 
Administrator Appraisal System and the TPDS can be continually monitored and improved. 
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School-to-Career Programs 

Program Director: 

Evaluation Staff: 

Program Description 

School-to-Career describes a system of integrated school- and work-based learning 
that integrates academic and occupational learning. Students are trained through a series of 
organized educational programs. These programs offer a sequence of courses that prepare 
students in paid or unpaid employment for current or emerging occupations. The Carl D. 
Perkins Vocational and Applied Technology Education Act (P.L. 101-392) specifically assists 
states to expand and improve their programs of vocational education and provide equal 
access in vocational education to special populations. As amended in 1990, the Perkins Act 
focuses on increasing the participation of special populations, and on improving the quality 
of vocational programs in schools with the greatest need for improvement. 

Evaluation Objectives 

1 . To comply with federal law requiring an annual evaluation of programs funded with 
Carl Perkins Act monies; and 

2. To provide information for decisionmakers on program effectiveness to facilitate 
decisions about program modification. 

Scope and method 

During the 1997-98 school year, OPE staff conducted several evaluation activities to 
provide baseline data to staff of the Department of School-to-Career Programs. During the 
1998-99 school year, the Office of Program Evaluation will undertake a variety of activities 
to provide evaluation information and, in many cases, longitudinal data concerning specific 
school-to-career initiatives. A feedback will be produced and distributed that summarizes 
evaluation activities associated with the programs discussed below. 

Counseling 

A survey will be conducted with graduating seniors to determine their overall 
satisfaction with the career counseling program. The survey will be conducted prior to 
Christmas break. A small sample of counselors, principals, and graduating seniors will 
participate in focus groups to obtain qualitative data concerning the career counseling 
program. The interviews and focus groups will be conducted in January and February 1999. 

Questions will be included on the district's Employee Coordinated Survey to obtain 
information from a sample of counselors, teachers, and administrators about the 
effectiveness of the career counseling program. 

Cooperative Education 

Students participating in the cooperative education program will be surveyed to 
obtain information about the relationship between their career goals, coursework, and the 
requirements of their present job. In addition, information will be obtained from students and 
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employers regarding their general satisfaction with the cooperative education program. Data 
will be gathered before the end of the first semester. 

Carl Perkins Grant 

OPE evaluation staff will assist School-to-Career staff to complete reporting 
requirements to TEA. In addition, OPE staff will work with School-to-Career staff on ways 
to identify career pathways students, along with other subgroups within the career and 
technology program. Once a reliable database is established, information requests and 
reporting requirements can be addressed more efficiently. The database will also be used to 
address the department’s goal of integrating academic classes with career and technology 
courses. 

College Bound 

Students on the college bound track will be followed to determine if the program goal 
of students attending college is met. The State Occupational Information Coordinating 
Committee (SOIC) data will be used in future years to track former students. Test results 
(PS AT, TAAS) for students taking the AP exam will be compared with results for students 
not taking the AP exam. Merit scholars and non-merit scholars test results will be compared 
as well to investigate differences in these groups. 
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Additional Duties of the Office of Program Evaluation, 1998-99 
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Ad hoc Requests and Special Projects 



Evaluation Staff: 



Holly Williams, Holly Koehler, 
Ralph Smith, Karen Wendt 



Description 

During the school year, the Office of Program Evaluation receives additional requests 
for assistance from district administrators and campuses. Projects range from literature 
reviews and program information for the school board to student data requests and assistance 
with evaluations of campus projects. It is our hope that if you have needs of this kind, you 
have already informed us by sending in a Request for Evaluation Assistance Form. If you 
have not sent in your request but already know your needs, please call Lillian Ray at 414- 
3641 and ask that she send you a Request for Evaluation Assistance Form. Complete this 
form and return it to Holly Williams as soon as possible. If you have a need that occurs 
during the school year, please contact Holly Williams as soon as possible. Staff in the Office 
of Program Evaluation will do their best to work your request into the full agenda and to 
respond to your need in a timely manner. 
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Employee Coordinated Survey 

Evaluation Supervisor: Ralph Smith 

Evaluation Staff: Karen Wendt 

Program Description 

Beginning in the 1979-80 school year, OPE formerly the Office of Research and 
Evaluation (ORE) conducted survey research with students, employees, and parents as a 
means of collecting opinion data from those with the greatest immediate stake in the district. 
By providing a vehicle for district stakeholders to reflect their opinion about key issues, 
surveys provide information about the effectiveness of programs and improvement activities 
implemented in AISD. Survey results are useful to district administrators in gaining insight 
into the strengths and weaknesses of various programs and policies of interest. 

In the early 1 980s, the Office developed and refined a sophisticated employee survey 
system that permitted sampling of survey items across respondents, thus providing the means 
to collect information on a wide variety of district programs and activities while minimizing 
the paperwork burden on teachers and other staff. The number of items an individual teacher 
received was capped at 24, for example. The survey system also permitted specific items to 
be targeted to specific respondent groups, such as special education teachers, or to a random 
sample of respondents. The annual survey replaced multiple, separate evaluation data 
collections and also afforded a vehicle to other district staff to gather opinion data 
efficiently. Through the mid-1980s and into the early 1990s, teachers, other professionals, 
and administrators were surveyed annually about specific programs, as well as on a broad 
range of topics of general interest, such as school climate, instructional television, dropout 
prevention efforts, parent involvement, and teacher career goals. Surveys were administered 
during a faculty meeting, resulting in percentages of surveys returned in the high 90s. 

The Employee Coordinated Survey was first administered during the 1992-93 school 
year. This survey differed from previous years’ employee surveys in that items were no 
longer sampled across staff. Instead, single-topic surveys were sent to respondents targeted 
according to the instructions from the staff submitting survey items (e.g., all bilingual/ESL 
teachers or a 20% random sample of all teachers). Coordination ensured that no teacher 
received more than one program’s survey. Separate instructions and information provided by 
evaluation staff were inserted into individual envelopes which were mailed directly to each 
administrator or campus professional. Reminders were sent out, and duplicate surveys were 
sent as requested. There was a 78% overall return rate. Subsequent administrations of the 
Coordinated Survey have followed nearly the same procedures, with similar return rates. 
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Evaluation Objectives 

1. To serve as a vehicle for OPE staff to gather opinion information as part of the 
evaluation of programs; 

2. To provide district administrators with a means to obtain information about various 
programs and policies of interest; and 

3. To replace multiple, separate data collections that might otherwise occur with a 
single, coordinated data collection which minimizes the paperwork burden on teachers 
and other staff. 

Scope and Method 

The 1998-99 Employee Coordinated Survey will be administered in March 1999 by 
the Office of Program Evaluation (OPE). The survey will follow the same procedures used in 
the previous four years. Items will again be solicited from both instructional and 
administrative support staff, as well as OPE staff. Teachers, other professionals, and 
administrators will be surveyed to answer questions related to the evaluations of federal Title 
programs and other programs, special education, gifted education, and other topics. As much 
as possible within the limits of the system, respondents will be surveyed according to 
instructions from the staff submitting the survey items (e.g., 20% random sample of all 
teachers). As in the past, individual envelopes with the respondent’s name on a removable 
label will be sent out. To secure a higher return rate, principals will be requested to administer 
the surveys during a faculty meeting. In addition, an instruction sheet, which includes a 
contact person’s name and phone number, will be inserted into the envelope with each 
survey. The anticipated return rate for the survey is from 70% to 90%. For purposes of 
survey administration, individual respondents will not be anonymous, but the confidentiality 
of their responses will be protected. Only aggregate data will be reported. When the survey 
processing has been completed, the computer files linking responses to individuals will be 
erased. Survey results will be analyzed and reported as follows: 

1 . For each survey item, number of responses will be tabulated for each response option 
(e.g., “strongly agree,” “agree,” etc.) by type of respondent (i.e., teacher, 
administrator, and other professional) and by level, (i.e., elementary, middle/junior 
high school, and high school). 

2. For each survey item, the number of surveys sent, returned, invalid/blank, and valid 
will be reported. 

3. Survey results will be reported by category (e.g., the survey items related to bilingual 
education). 

4. Aggregate results will be returned to the persons who submitted the survey items (e.g., 
the results of special education items to the director of Special Education). 

5. A complete set of results will be maintained on file in OPE, along with work papers 
(communications, printouts, etc.) detailing the survey process. 
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Internal Data Requests 

Evaluation Staff: Ralph Smith, Karen Wendt 

Internal Data requests by AISD personnel will be performed as needed. These differ 
slightly from ad hoc requests in that they are mainly requests for data. In response to these 
requests, data are compiled and then returned with a brief explanation of the data to the 
person making the request. 

Office of Program Evaluation Web Page 

Evaluation Staff: Ralph Smith 

During the 1997-98 school year, a web page was developed for the Office of Program 
Evaluation. During the 1998-99 school year, this web page will be updated as needed to 
provide current information about the Office of Program Evaluation (OPE). Information 
provided on the web page includes: the Office of Program Evaluation agenda describing 
current research projects; staff contacts for OPE projects; OPE research reports available to 
the public; executive summaries of OPE reports that provide project overviews; and Internet 
links to other educational sites. The OPE web page can be found at 
http://www.austin.isd.tenet.edu/admin/ope, or as link from the AISD home page. 

Procedures Manual 

Evaluation Staff: Holly Williams, Ralph Smith, Holly Koehler, Gloria 

Zyskowski, Martha Doolittle, Michelle Batchelder 

The Procedures Manual for the Office of Program Evaluation (OPE) will be revised 
to reflect current changes in district/office Administration, Finance, Personnel, Data 
Collection, Data Management, Typing Guides, Publications, and Files. 

SAS 

Evaluation Staff: Holly Koehler, Holly Williams 

During the 1998-99 school year, the Office of Program Evaluation will be 
incorporating the use of SAS statistical analysis software. SAS is software that will help 
evaluate the effectiveness of programs; it can be used to perform multiple complex statistical 
analyses used in research and evaluation. 
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